Efficient autonomous learning for statistical pattern recognition

We describe a neural network learning algorithm that implements differential learning in a generalized backpropagation framework. The algorithm regulates model complexity during the learning procedure, generating the best low-complexity approximation to the Bayes-optimal classifier allowed by the training sample. It learns to recognize handwritten digits of the AT&T DB1 database. Learning is done with little human intervention. The algorithm generates a simple neural network classifier from the benchmark partitioning of the database; the classifier has 650 total parameters and exhibits a test sample error rate of 1.3%.